Skip to content

⬆️🔒️ Maintenance/vulnerability upgrade for ujson, upgrade fastapi+starlette#3112

Merged
pcrespov merged 9 commits intoITISFoundation:masterfrom
pcrespov:maintenance/vulnerability-ujson
Sep 20, 2022
Merged

⬆️🔒️ Maintenance/vulnerability upgrade for ujson, upgrade fastapi+starlette#3112
pcrespov merged 9 commits intoITISFoundation:masterfrom
pcrespov:maintenance/vulnerability-ujson

Conversation

@pcrespov
Copy link
Copy Markdown
Member

@pcrespov pcrespov commented Jun 16, 2022 •
edited
Loading

What do these changes do?

ON HOLD -> recently released https://github.com/tiangolo/fastapi/releases/tag/0.85.0

Related issue/s

How to test

  • services/api-server/tests/unit/test__fastapi.py

Checklist

Highlights on updated libraries (only updated libraries are included)

  • #packages before: 3
  • #packages after : 3
# name before after upgrade count packages
1 fastapi 0.71.0, 0.75.0, 0.75.1, 0.82.0 0.85.0 minor 7 api-server⬆️
autoscaling⬆️
catalog⬆️
datcore-adapter⬆️
director-v2⬆️
dynamic-sidecar⬆️
service-library🧪
2 starlette 0.19.1, 0.17.1 0.20.4 minor 7 api-server⬆️
autoscaling⬆️
catalog⬆️
datcore-adapter⬆️
director-v2⬆️
dynamic-sidecar⬆️
service-library🧪
3 ujson 4.3.0, 5.3.0 5.5.0 minor 5 api-server⬆️
catalog⬆️
director-v2⬆️
storage⬆️
web⬆️

Legend:

  • ⬆️ base dependency (only services because packages are floating)
  • 🧪 test dependency
  • 🔧 tool dependency

Repo-wide overview of libraries

  • #reqs files parsed: 60
# name versions-base versions-test versions-tool
1 aio-pika 6.8.0, 7.2.0 6.8.0, 8.2.0
2 aioboto3 9.6.0 10.0.0
3 aiobotocore 2.3.0, 2.3.3 2.3.4
4 aiocache 0.11.1 0.11.1
5 aiodebug 2.3.0 2.3.0
6 aiodocker 0.19.1, 0.21.0 0.21.0
7 aiofiles 0.8.0, 22.1.0 22.1.0
8 aiohttp 3.8.1 3.8.1
9 aiohttp-jinja2 1.5
10 aiohttp-security 0.4.0
11 aiohttp-session 2.11.0
12 aiohttp-swagger 1.0.16
13 aioitertools 0.10.0 0.10.0
14 aiopg 1.3.3, 1.3.4 1.3.4
15 aioredis 2.0.1
16 aioresponses 0.7.3
17 aiormq 3.3.1, 6.2.3 3.3.1, 6.4.1
18 aiosignal 1.2.0 1.2.0
19 aiosmtplib 1.1.6
20 aiozipkin 1.1.1
21 alembic 1.8.1 1.8.1
22 anyio 3.5.0, 3.6.1 3.5.0, 3.6.1
23 argon2-cffi 20.1.0
24 asgi-lifespan 1.0.1
25 asgiref 3.5.0, 3.5.2
26 astroid 2.12.9 2.12.9
27 async-asgi-testclient 1.4.11
28 async-generator 1.10
29 async-timeout 4.0.2 4.0.2
30 asyncpg 0.25.0
31 attrs 21.4.0, 22.1.0 21.4.0, 22.1.0
32 aws-sam-translator 1.50.0
33 aws-xray-sdk 2.10.0
34 bcrypt 3.2.0 4.0.0
35 beautifulsoup4 4.10.0
36 black 22.8.0
37 bleach 3.3.0
38 blosc 1.10.6
39 bokeh 2.4.3 2.4.3
40 boto3 1.21.21, 1.21.33 1.21.21, 1.24.67
41 boto3-stubs 1.24.67
42 botocore 1.24.21, 1.24.33 1.24.21, 1.27.67
43 botocore-stubs 1.27.17 1.27.67
44 build 0.8.0
45 bump2version 1.0.1
46 certifi 2021.10.8, 2022.5.18.1, 2022.6.15, 2022.6.15.1 2021.10.8, 2022.5.18.1, 2022.6.15, 2022.6.15.1
47 cffi 1.15.0 1.15.0, 1.15.1
48 cfgv 3.3.1
49 cfn-lint 0.64.1
50 change-case 0.5.2
51 charset-normalizer 2.0.12, 2.1.1 2.0.12, 2.1.1
52 click 8.1.3 8.1.3 8.1.3
53 cloudpickle 2.0.0, 2.1.0
54 codecov 2.1.12
55 colorlog 6.7.0
56 configparser 5.2.0
57 coverage 6.4.4
58 coveralls 3.3.1
59 cryptography 3.4.7, 36.0.2, 37.0.2 36.0.2, 38.0.0
60 cytoolz 0.11.0
61 dask 2022.6.0, 2022.9.0
62 dask-gateway 2022.6.1
63 dask-gateway-server 2022.6.1
64 decorator 4.4.2
65 defusedxml 0.7.1
66 deprecated 1.2.13 1.2.13
67 dill 0.3.5.1 0.3.5.1
68 distlib 0.3.6
69 distributed 2022.6.0, 2022.9.0
70 distro 1.5.0
71 dnspython 2.0.0, 2.1.0, 2.2.1 2.2.1
72 docker 5.0.3, 6.0.0 6.0.0
73 docker-compose 1.29.1
74 dockerpty 0.4.1
75 docopt 0.6.2 0.6.2
76 ecdsa 0.14.1 0.18.0
77 email-validator 1.2.1 1.2.1
78 entrypoints 0.3
79 et-xmlfile 1.1.0
80 exceptiongroup 1.0.0
81 execnet 1.9.0
82 expiringdict 1.2.1
83 faker 14.2.0
84 fastapi 0.85.0
85 fastapi-contrib 0.2.11
86 fastapi-pagination 0.9.1
87 fastjsonschema 2.15.3
88 filelock 3.8.0
89 flaky 3.7.0
90 flask 2.1.3
91 flask-cors 3.0.10
92 frozenlist 1.3.0, 1.3.1 1.3.0, 1.3.1
93 fsspec 2022.5.0, 2022.8.2
94 future 0.18.2
95 futures 3.0.5
96 graphql-core 3.2.1
97 greenlet 1.1.2, 1.1.3 1.1.2, 1.1.3
98 gunicorn 20.1.0
99 h11 0.12.0 0.12.0
100 h2 4.1.0
101 heapdict 1.0.1
102 hpack 4.0.0
103 httpcore 0.15.0 0.15.0
104 httptools 0.2.0, 0.4.0
105 httpx 0.23.0 0.23.0
106 hyperframe 6.0.1
107 hypothesis 6.54.5
108 icdiff 2.0.5
109 identify 2.5.5
110 idna 2.10, 3.3 2.10, 3.3
111 importlib-metadata 4.12.0
112 iniconfig 1.1.1 1.1.1
113 inotify 0.2.10
114 isodate 0.6.1
115 isort 5.10.1 5.10.1
116 itsdangerous 1.1.0, 2.1.2 2.1.2
117 jaeger-client 4.8.0
118 jinja-app-loader 1.0.2
119 jinja2 3.1.2 3.1.2 3.1.2
120 jmespath 1.0.0 1.0.0, 1.0.1
121 jschema-to-python 1.2.3
122 json2html 1.3.0
123 jsondiff 2.0.0 2.0.0
124 jsonpatch 1.32
125 jsonpickle 2.2.0
126 jsonpointer 2.3
127 jsonschema 3.2.0, 4.15.0 3.2.0, 4.15.0
128 junit-xml 1.9
129 jupyter-client 6.1.12
130 jupyter-core 4.7.1
131 jupyter-server 1.18.1
132 jupyter-server-proxy 3.2.1
133 jupyterlab-pygments 0.1.2
134 lazy-object-proxy 1.7.1 1.7.1 1.7.1
135 locket 1.0.0
136 lz4 4.0.0
137 mako 1.1.5, 1.2.0, 1.2.2 1.1.5, 1.2.0, 1.2.2
138 markupsafe 2.1.1 2.1.1 2.1.1
139 mccabe 0.7.0 0.7.0
140 minio 7.0.4
141 mistune 0.8.4
142 moto 4.0.1, 4.0.2
143 msgpack 1.0.3, 1.0.4
144 multidict 6.0.2 6.0.2
145 mypy-extensions 0.4.3
146 nbclient 0.5.3
147 nbconvert 6.4.5
148 nbformat 5.3.0
149 nest-asyncio 1.5.1
150 networkx 2.5.1 2.8.6
151 nodeenv 1.7.0
152 nose 1.3.7
153 numpy 1.22.3 1.23.2
154 openapi-core 0.12.0
155 openapi-schema-validator 0.2.3 0.2.3
156 openapi-spec-validator 0.4.0 0.4.0
157 openpyxl 3.0.9
158 opentracing 2.4.0
159 orjson 3.7.2
160 packaging 21.3 21.3 21.3
161 pamqp 2.3.0, 3.1.0 2.3.0, 3.2.0
162 pandas 1.2.4 1.4.4
163 pandocfilters 1.4.3
164 paramiko 2.11.0
165 parfive 1.5.1
166 partd 1.2.0, 1.3.0
167 passlib 1.7.4 1.7.4
168 pathspec 0.10.1
169 pbr 5.10.0
170 pennsieve 6.2.0
171 pep517 0.13.0
172 pillow 9.0.1 9.2.0
173 pint 0.19.2 0.19.2
174 pip-tools 6.8.0
175 platformdirs 2.5.2 2.5.2
176 pluggy 1.0.0 1.0.0
177 pprintpp 0.4.0
178 pre-commit 2.20.0
179 prometheus-client 0.14.1
180 protobuf 3.20.0
181 psutil 5.9.0, 5.9.1, 5.9.2
182 psycopg2-binary 2.9.3 2.9.3
183 ptvsd 4.3.2
184 ptyprocess 0.7.0
185 py 1.11.0 1.11.0
186 py-cpuinfo 8.0.0
187 pyasn1 0.4.8 0.4.8
188 pycparser 2.20, 2.21 2.20, 2.21
189 pydantic 1.9.0, 1.10.2 1.10.2
190 pyftpdlib 1.5.6
191 pygments 2.9.0
192 pyinstrument 3.4.2, 4.1.1, 4.3.0 4.3.0
193 pyinstrument-cext 0.2.4
194 pyjwt 2.4.0
195 pylint 2.15.0, 2.15.2 2.15.0
196 pynacl 1.4.0
197 pyopenssl 22.0.0
198 pyparsing 3.0.9 3.0.9 3.0.9
199 pyrsistent 0.18.1 0.18.1
200 pytest 7.1.3 7.1.3
201 pytest-aiohttp 1.0.4
202 pytest-asyncio 0.19.0
203 pytest-benchmark 3.4.1
204 pytest-cov 3.0.0
205 pytest-docker 1.0.0
206 pytest-forked 1.4.0
207 pytest-icdiff 0.6
208 pytest-instafail 0.4.2
209 pytest-lazy-fixture 0.6.3
210 pytest-localftpserver 1.1.3
211 pytest-mock 3.8.2
212 pytest-runner 6.0.0
213 pytest-sugar 0.9.5
214 pytest-xdist 2.5.0
215 python-dateutil 2.8.1, 2.8.2 2.8.1, 2.8.2
216 python-dotenv 0.20.0 0.20.0, 0.21.0
217 python-engineio 3.14.2
218 python-jose 3.2.0 3.3.0
219 python-magic 0.4.25
220 python-multipart 0.0.5
221 python-socketio 4.6.1
222 pytz 2020.1, 2022.1 2022.2.1
223 pyyaml 5.4.1, 6.0 5.4.1, 6.0 5.4.1, 6.0
224 pyzmq 22.1.0
225 redis 4.3.1, 4.3.4 4.3.1
226 requests 2.27.1, 2.28.1 2.27.1, 2.28.1
227 responses 0.21.0
228 respx 0.19.2
229 rfc3986 1.4.0, 1.5.0 1.4.0, 1.5.0
230 rsa 4.9 4.9
231 s3fs 2022.5.0
232 s3transfer 0.5.2 0.5.2, 0.6.0
233 sarif-om 1.0.4
234 semantic-version 2.9.0
235 semver 2.13.0
236 send2trash 1.7.1
237 setproctitle 1.2.3
238 simpervisor 0.4
239 six 1.15.0, 1.16.0 1.15.0, 1.16.0
240 sniffio 1.2.0, 1.3.0 1.2.0, 1.3.0
241 sortedcontainers 2.4.0 2.4.0
242 soupsieve 2.3.2
243 sqlalchemy 1.4.37, 1.4.41 1.4.37, 1.4.41
244 sshpubkeys 3.3.1
245 starlette 0.20.4
246 strict-rfc3339 0.7
247 tblib 1.7.0
248 tenacity 8.0.1 8.0.1
249 termcolor 1.1.0
250 terminado 0.10.1
251 testpath 0.5.0
252 texttable 1.6.3
253 threadloop 1.0.2
254 thrift 0.16.0
255 toml 0.10.2
256 tomli 2.0.1 2.0.1 2.0.1
257 tomlkit 0.11.4 0.11.4
258 toolz 0.11.1, 0.12.0
259 tornado 6.1, 6.2 6.1
260 tqdm 4.64.0, 4.64.1 4.64.1
261 traitlets 5.1.1 5.3.0
262 twilio 7.12.0
263 typer 0.4.1, 0.6.1 0.6.1 0.6.1
264 types-aiobotocore 2.3.3
265 types-aiobotocore-s3 2.3.3
266 types-aiofiles 0.8.11
267 types-awscrt 0.14.5
268 types-boto3 1.0.2
269 types-pkg-resources 0.1.3
270 types-pyyaml 6.0.11
271 types-s3transfer 0.6.0.post4
272 typing-extensions 4.3.0 4.3.0 4.3.0
273 ujson 5.5.0
274 urllib3 1.26.9, 1.26.11, 1.26.12 1.26.9, 1.26.11, 1.26.12
275 uvicorn 0.15.0, 0.17.0, 0.17.6, 0.18.3
276 uvloop 0.16.0
277 virtualenv 20.16.4, 20.16.5
278 watchdog 2.1.5 2.1.9
279 watchgod 0.8.2
280 webencodings 0.5.1
281 websocket-client 0.59.0, 1.3.2, 1.4.1 0.59.0, 1.4.1
282 websockets 10.1, 10.2 10.3
283 werkzeug 2.0.3, 2.1.2 2.0.3, 2.1.2
284 wheel 0.37.1
285 wrapt 1.14.0, 1.14.1 1.14.0, 1.14.1 1.14.1
286 xmltodict 0.13.0
287 yarl 1.5.1, 1.7.2, 1.8.1 1.5.1, 1.7.2, 1.8.1
288 zict 2.2.0
289 zipp 3.8.1

@pcrespov pcrespov self-assigned this Jun 16, 2022
@pcrespov pcrespov changed the title ⬆️ Maintenance/vulnerability upgrade for ujson ⬆️🔒️ Maintenance/vulnerability upgrade for ujson Jun 16, 2022
@pcrespov pcrespov added dependencies security Pull requests that address a security vulnerability labels Jun 16, 2022
@pcrespov pcrespov added this to the Diolkos milestone Jun 16, 2022
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 16, 2022 •
edited
Loading

Codecov Report

Merging #3112 (b49c4cf) into master (988cc25) will increase coverage by 0.0%.
The diff coverage is 100.0%.

Impacted file tree graph

@@          Coverage Diff           @@
##           master   #3112   +/-   ##
======================================
  Coverage    82.9%   83.0%           
======================================
  Files         808     808           
  Lines       34261   34261           
  Branches     1358    1358           
======================================
+ Hits        28419   28450   +31     
+ Misses       5661    5630   -31     
  Partials      181     181
Flag Coverage Δ
integrationtests 68.5% <ø> (+0.1%) ⬆️
unittests 79.8% <100.0%> (+<0.1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...re_service_datcore_adapter/modules/remote_debug.py 56.2% <100.0%> (ø)
...webserver/computation_comp_tasks_listening_task.py 81.9% <0.0%> (-3.2%) ⬇️
...rc/simcore_service_catalog/db/repositories/dags.py 44.4% <0.0%> (-2.8%) ⬇️
...simcore_service_director_v2/modules/node_rights.py 98.1% <0.0%> (-1.0%) ⬇️
...ore_service_director_v2/utils/dask_client_utils.py 80.8% <0.0%> (-0.9%) ⬇️
.../director/src/simcore_service_director/producer.py 67.5% <0.0%> (+0.2%) ⬆️
...simcore_service_director_v2/modules/dask_client.py 93.4% <0.0%> (+0.5%) ⬆️
...tor_v2/modules/dynamic_sidecar/docker_api/_core.py 95.7% <0.0%> (+0.9%) ⬆️
...tor_v2/modules/dynamic_sidecar/scheduler/_utils.py 90.8% <0.0%> (+1.1%) ⬆️
...rector_v2/modules/comp_scheduler/base_scheduler.py 92.4% <0.0%> (+1.2%) ⬆️
... and 8 more

@pcrespov pcrespov force-pushed the maintenance/vulnerability-ujson branch 3 times, most recently from 2f17666 to 39d9f2e Compare June 22, 2022 16:04
@sanderegg sanderegg mentioned this pull request Jul 3, 2022
Closed
2 tasks
@pcrespov pcrespov force-pushed the maintenance/vulnerability-ujson branch from 0598c50 to 7876149 Compare July 7, 2022 10:39
@pcrespov pcrespov modified the milestones: Diolkos, Meteora Jul 7, 2022
@pcrespov pcrespov force-pushed the maintenance/vulnerability-ujson branch from 7876149 to 50b01da Compare July 7, 2022 15:50
@pcrespov pcrespov changed the title ⬆️🔒️ Maintenance/vulnerability upgrade for ujson WIP ⬆️🔒️ Maintenance/vulnerability upgrade for ujson Jul 7, 2022
@pcrespov pcrespov force-pushed the maintenance/vulnerability-ujson branch 2 times, most recently from 41453bb to a8399d0 Compare July 12, 2022 13:19
@pcrespov pcrespov changed the title WIP ⬆️🔒️ Maintenance/vulnerability upgrade for ujson WIP ⬆️🔒️ Maintenance/vulnerability upgrade for ujson (ON HOLD) Jul 12, 2022
@pcrespov pcrespov force-pushed the maintenance/vulnerability-ujson branch from a8399d0 to 3e4e8c3 Compare July 18, 2022 21:04
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Jul 18, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@pcrespov pcrespov force-pushed the maintenance/vulnerability-ujson branch from 3e4e8c3 to 36b7832 Compare September 18, 2022 12:30
@pcrespov pcrespov changed the title WIP ⬆️🔒️ Maintenance/vulnerability upgrade for ujson (ON HOLD) ⬆️🔒️ Maintenance/vulnerability upgrade for ujson Sep 18, 2022
@pcrespov pcrespov changed the title ⬆️🔒️ Maintenance/vulnerability upgrade for ujson ⬆️🔒️ Maintenance/vulnerability upgrade for ujson, upgrade fastapi+starlette Sep 18, 2022
@pcrespov pcrespov force-pushed the maintenance/vulnerability-ujson branch from 36b7832 to de347a6 Compare September 19, 2022 11:58
@pcrespov pcrespov modified the milestones: Meteora, vaporwave Sep 19, 2022
@pcrespov pcrespov marked this pull request as ready for review September 19, 2022 11:58
@pcrespov pcrespov force-pushed the maintenance/vulnerability-ujson branch from 7906918 to b145096 Compare September 19, 2022 14:09
GitHK
GitHK approved these changes Sep 20, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment thread services/api-server/tests/unit/test__fastapi.py Outdated
@pcrespov pcrespov force-pushed the maintenance/vulnerability-ujson branch 2 times, most recently from d7d04b6 to 56255a0 Compare September 20, 2022 14:53
@pcrespov pcrespov force-pushed the maintenance/vulnerability-ujson branch from 56255a0 to b49c4cf Compare September 20, 2022 18:11
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Sep 20, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@pcrespov pcrespov merged commit 1e08132 into ITISFoundation:master Sep 20, 2022
@pcrespov pcrespov deleted the maintenance/vulnerability-ujson branch September 20, 2022 21:44
@sanderegg sanderegg mentioned this pull request Oct 5, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@GitHK GitHK GitHK approved these changes

@mrnicegyu11 mrnicegyu11 mrnicegyu11 approved these changes

@mguidon mguidon mguidon approved these changes

@sanderegg sanderegg sanderegg approved these changes

@Surfict Surfict Awaiting requested review from Surfict

Assignees

@pcrespov pcrespov

Labels

security Pull requests that address a security vulnerability

Projects

None yet

Milestone

vaporwave

Development

Successfully merging this pull request may close these issues.

5 participants

@pcrespov @GitHK @mrnicegyu11 @mguidon @sanderegg